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ABSTRACT 

This document defines cooperative education's an] 
form of occupational or prof essional activity that required Vhe 
cooperation of both school and the labor market. In some cases, this 
might be the school and industry :or business. In this process, 
evaluation is defined as the improvement of learner success through 
measurement of program components e Stress is placed on translating 
evsCluational programs into documentation of individual learner 
success. This program approach tries to cut across courses in order 
to stress growth that is both upward and cumulative. This growth 
process is contrasted with growth that is horizontal and additive in 
the sense of more of the same rather than more complexity/ and depth. 
Twelve learning environments are also provided which giv| short and 
simple approaches to a humanistic evaluation of cooperative education 
programs. (iuthor/DEP) / 
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EVALUATING COOPERATIVE EDUCATION HEALTH PROGRAMS 

This document defines cooperative education as any form of occupational 
or^-professional activity that requires the cooperation of both school;and 
the labor market. In some cases, this might be the school and industry or 
business. 

In this process, evaluation is defined as the improvement of learner 
success "through measurement of program components. ■ • 

Stress is placed upon translating evaluational programs into dociimentation 

of individual learner success. ^ 

This program approach tries to cut across courses in order to stress 
growth that i3 both upward and cumulative. This growth process is contrasted: 
with growth that is horizontal and additive in the sense of more of the same, 
rather than more complfcxity and depth. 



In too many schools, evaluation simply means a few faculty 'm^nibers 

\ 

filling out two or three forms. Seldom is teacher behavior changed. 

Evaluation^ is not a-means of forcing people to change* Evaluation 

helps people change by providing necessary assistance* 

This document tries to link real evaltiatlon procedures with cooperative 

education in such a way as to aim at specif ic and measurable outcomes of 

evaluation* 
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HUMANISTIC EVALUATION 



From a hunanlstlc point of view, failttre to evaltiate the results of 
cooperative education is posaibly one of the cdnmonest of all faults of 
such programs. Structuring the cooperative education program is difficult , 
but this is not enough No matter how well planned , scheduledj^ and 
conducted the program Is, the entire cooperative program must be evaluated. 

In other words, someona i^ st ascertain whether the cooperative eduction 
education program was successful. Cooperative education is so vast an 
undertaking that merely asking students and teachers how the program was 
liked or disliked is not enough. The real question of evaluation goes 
beyond surveying opinions about how interesting, boring, stimulating, or 
tiresome the Qooperatlve education program wa^^^ 

The real evaluation question is, *Vhat did each participant employee 
learn?" No matter how splendid the sessions were, no matter ho/ well * 
the meetings were conducted, or no matter how scholarly were the 
presentations, the ln^ortant point is to find out what each individual 
learned. Any experienced cooperative educator has come up with a number 
of evaluation techniques that work well and are low cost. The following 
are presented to sho«r that no magic fwraila is needed* 

l# Knowledge evaluaticm can be achieved through oral or 
written tests given to find out whether or not the 
trainee has learned what has been told or given as 
a reading assignment. Such tests are best when 
kept simple and uhelaborate. These tests should 
establish beyond any serious doubt whether the person 
in training has the required knowledge. 



2* Performance evaluation Is something that can be 
observed by the teacher or trainer to determine 
whether the trainee has developed the skill to 
do the required vork« The biggest difficulty 
- teachers have here Is tO: avoid coaching the 
_ learner during performance evaluation • This 
type of skill evaluation imist evaltiate the 
learner working Independent lyo - _ 

3. Attitude evaluation is quite different from 
psychoanalysis in the sense that attitude 
evaluation looks at attitude indicators ^ not 
interior and invisible feelings ♦ The work 
done by a person in a cooperative education 
program should be reviewed frcna time to time 
in order to check quantity and quality of 
results^ This type of evaluation ordinarily, 
comes aifter knox^ledge evaluation and performance ^ 
evaluation as outlined above« 

Many people have been asked t "Is your cooperative education progr^ 
worthwhile?" ' 

A complicated reply to this question may be based upon statistical . 
analjrsis of a wide variety of testing Instruments • Howevert the person , 
who asks this question wants a imich more simple and direct response* 
Anyone who answers this question with, *'Our graduates like the program* 
Many of the employers want to keep the cpoperatlve education students on 
the Job after graduation* As an Inducement for this, many of these 
cooperative education employers give the student full credit for the 
training period* This means that the beginning worker, after graduation, 
is able to start with a salary Increment equal to 6 months on the iob*" 

The above quotation can be used be anyone who has developed a successful 
cooperative education program* The case history given by the specific 
evaluator will vary from institution to institution* However, whenever 
the cooperative education program is able to point to specific results 
in terms of ^loyablllty, the evaluation has gone beyond the realm of 
statistics and entered a very humanistic dlmehslon* , 




In some vays^ the following document is quite technical* This is 
necessary because o£ the requirements of evaluating cooperative education 
programs • No nvatter how technical this doctmient may become^ it is hoped, 
that jreaders eiit)loying It will recall the very human applications stressed 
above* ' 

There is nothing magic about the evaluation system herein proposed^ 
It siii^)ly enables learners and teachers to keep score in such a way as to 
strive for daily and long term progress* 
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1 APPLICATION WORKSHOPS 

During the months of July to October, 1974, the preceding document 
: was used at a number of workshops * These workshops were intended for 
evaluators of cooperative education* 

In addition tq stressing cooperative education evaluation, each of ^ 
these workshops also went into the area of evaluating inservice education. 
After a while, it was noticed that the similarity of the measurement 
problems was enough to justify several common endeavors* 

In order to provide to the evaluator of cooperative education a 
ntunber of practical training activities, the following learning 
envjbronments are appended.beginning on page 26. 

It is hoped that by going through these learning environments, the: 
beginning evaluator will be able to systematize the evaluation task in 
such a fashion as to provide a comprehensive and systematic approach. 

The common thread that links together these learning environments 
with the preceding document on evaluating cooperative education is the 
; stress on learner development* In other words, the basic evaluation 
flwstion is, '*What documented evidence do we have that the cooperative 
education program or the inservice training program has produced a 
measurable result and impact on learners 2" 
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OBJECTIVES OT EVALmTING C(X)PERATIVE education; PROGRAMS 

----- " ^ - ^ .. .. 

Without attempting to provUe ai exhaustive list of objectives for 
evaluating cooperative education programs^ the following purposes will 
give a good general overview* This type of overview will enable educators 
and students to tsxcn out the type of evaluation which can provide iraplementable 
guidelines for progress* 

The basic objectives of evaltiating cooperative education programs are 



to: 



1. Provide an activity-by-activity evaluation that identifies 
objectives 9 time period ^ resources , personnel » and success 
of outcoae* \ < 

2. Identify activities and or ganizationa 
contributed or detracted from the overall objective. . 

3. Measure the degree to which participants have or have 
not met the standards of acceptable achievement inherent 
in^ thcL^ogram objectives o : 

4. Pinpoint the extent to which major cooperative 
education activities enhanced or detracted^^fe^ 
program objectives a 

5# Specify the extent to which non-instructional program 
elements (library, material availability, location, 
management, housing, food, climate control, non-classroom 
interaction opportunities, registration procedures, stipend 
reimbursement, and learning contracts) facilitated learning # 

6. Report the results of evaltiation in a form that can be 
easily comprehended and applied; 
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BASELINE DAlA FOR EVALUATING COOPERATIVE EDUCATION 

Once a list o£ objectives have be» developed for evialtiatlng cooperative 

education^ It Is only natinral to ask about the baseline data. Baseline 

- - ^ V* 

data pinpoints the level of achievement of a particular program at Its ^* 



beginning, 

In or< er to develop accurate and easy to Implement collection of 
baseline data^ the follo«d.ng checklist Is provided. In order to use 
this cliGckllsty ask whether or not the cooperative education program 
does the following: - 

YES NO 1. State objectives In behavioral terins, 

YES NO 2, State objectives in terms of participant outcomes, 

YES NO 3, Specify terminal behaviors acceptable as documentation 
that cooperative education objectives have been met, 

YES NO k. Link each learning activity to one or more of the 
specific objectives, 

YES NO 5» Provide a minimum of one learning activity for ^/ ; 

each objective* " 

YES NO 6# Measure overall reaction to the program from (a) 
participants^ (b) staff, and (c) director* 

YES NO ?• Provide for spontaneous and creative input from 

participating students, staff, and administrators. 
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INTRODUCTION 



A HUMANISTIC APPROACH TO THE TECHNOLOGY 
OF EVALUATING COOPEPJ^TIVE EDUCATION PROGRAMS 



This article contains a reasonably complete how-to-do-it kit for 
would-be evaluators of cooperative education, Xwo parts subdivide 
this approach. 

Part 1 contains the author's implementation guidelines for 
evaluating cooperative education programs. Part 1 gives the three 
evaluation questions, three examples of these questions, and three 
conclusions about , practical implementation. 

Part 2 contains applications developed by cooperative educators 
who have been influenced by the author's approach. Part 2 summarizes 
evaluation expectations of cooperative educators. In addition, evaluation 
suggestions that have been found practical are provided. Both the expectancie 
and suggestions of cooperative educators are analyzed by a four level scale 
that results in the ability to document learner success due to a specific 
cooperative educational program components • In order to facilitate further 
application of this evaluation format by cooperative educators, a matrix 
analysis has been provided, A matrix is a simplified one page analysis 
that specifies objectives, evaluations, and resources in the three domains 
of knowledge, performance, and attitude, 
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PART 1 

THE ALVIR APPROACH TO EVALUATION 

Part 1 asks three questions that summarize local evaluation, gives 
three examples^ that show a would-be evaluator what to do, and draws 
three conclusions that help operationalize evaluation. 

In Xhis process, evaluation is considered as one means of improving 
program effectiveness. This accent on "one way" to improve effectiveness 
is an attempt to avoid rigidity, red tape, and monolithism. 

FLEXIBILITY is stressed in these three quests ors by allowing each 
participant' to express a sincere and independent approach. 

SIMPLICITY is stressed in these three questions by allowing each 
participant to clarify the local approach until it is crystal clear to 
others who have never heard about this specific adaptation. 

INDIVIDUALITY is stressed in these three questions by allowing 
several different approaches to exist side by side in cooperation and 
friendly competition rather than in conflict. 

It is easy to name the three questions: 

THE OBJECTIVE QUESTION 
THE EVALUATION QUESTION 

THE RESOURCE QUESTION - 

The objective question asks, "What is the precise benefit the learner 
is to achieve?" This question is intended to specify the educational 
target in specific terms. A more general statement of the purpose would 
be termed a goal rather than an objective. 
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The evaluation question asks, "How is this learner- success to be 
documented?" This question stresses tests and evaluation instruments 
that are judged to be effective yardsticks of learner progress and gains. 
The progress made by the learner between the beginning of the program and 
the end of the program is called a gains score. 

The resource question asks, "What are the alternatives open to the 
learner who wants to" achieve in a unique style?" This question provides 
technological backup to the learnev and the teacher. This approach 
assumes that there are many different ways to achieve an objective as 
measured by prespecified yardsticks. Each of these different approaches 
or resources has a unique price tag and rhythm of learning. 



These three questions should be answered in general by those 
responsible for a specific program. A general answer given without a 
long wait is one quick way to pinpoint priority material. Too much 
reflection given at this time will involve a number of lesser priority 
issues. Cluttering up the overall statement of a program's purpose with 
excessive detail is not the intention. of these questions. 
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LEVELS OF COOPERATIVE EDUCATION 

As a result of the objective question, several different levels of 
cooperative education emerge. These different levels constitute a nominal 
scale. 

Level 1 refers to general education work experience. This level takes 
the basic approach of learning and earning simultaneously. Very often, this 
level includes meeting once a week with a work experience counselor or with 
a cooperative general aided class that tovera many things any employee ^ 
would need to know such as taxes, withholdings, and fringe benefits. 

Level 2 is called career orientation work experience in the pre- vocational 
sense of the term. This includes such things as school to employment programs 
and work experience career educational exploration programs. The goals of 
these career orientation work experience programs are to prevent dropouts and 
to direct learners toward an occupational choice. 

Level 3 is called occupational cooperative education. This is the 
capstone or crowning experience approach to cooperative education. This level 
integrates school lab and related instruction with on-the-job training either 
in a single occupation or in diverse occupations. 

The above three levels constitute a nominal scale because each 

level has a different name. As far as level 3 is concerned, this is the 

precise terminology for a program that involves the three following elements: 

A. A training plan which spells. out step-by-step the expected ^ 
learner progress. 

•B. A written agreement that specifies- working conditions, minimum 

wage or better, and other training- agreements. 
. C. An educational system in which cooperative education means 
HIRE education. 

Cooperative education requires all three elements. 
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DOCUMENTING LEARNER SUCCESS 

The evaluation question provides a number of yardsticks that can be 
used to evaluate specific dimensions of cooperative educational success. 

Since cooperative education is not a different kind of learning, but 
a different way of learning, it is indeed realistic to apply the same 
yardsticks to cooperative education as are applied to any other educational 
approach. 

In addition, since cooperative education involves on-the-job 
training, -it is possible to include a number of other yardsticks in 
measuring its success: « 

A. Dollars earned oh the job ' 

B. Accuracy and mastery of marketable job skills * 

C. Personal satisfaction and fulfillment that accompany a 
good job well done " ; 

D. Productivity wherein the learner begins to take over 
the helm of a worthwhile experience 

In non- technical terms, it can be seen that certain yardsticks ^ 

emerge that give convincing proof of the practical value of cooperative 

education: 

^ YARDSTICK 1 

Many employers allow successful graduates of cooperative education 
the authorization to begin work at the first pay increment. In other words, 
cooperative educational programs have been evaluated by employers as better 
or equivalent to six months of on-the-job experience, 

YARDSTICK 2 

Many employers are sufficiently impressed with the quality of work 
done by cooperative education students as to continue a large number of these 
students on the same job at a better pay scale. This points out the fact that 
from the employers point of view, cooperative education has been successful. 

Er|c 17 



Any evaluator trying to assess the worthwhile benefits of cooperative 
education should consider yardsticks one and two before going on to 
COTipllcated statistical analysis. In addition to these obviously observable 
yardsticks, attention should be given to the ability for learners to learn 
better when what has been taught In the classroom Is seen, applied, and 
appreciated on the job. This approach to evaluation naturally goes Into 
the mergure between self-image and productivity. Psychoanalysis is not 
needed to measure self-image. It is sufficient to talk to learners about 
what they have absorbed while working on the job. 

It is not unusual to say about a project, "it was a great concept, 
but it hasn*t worked out," This negative evaluation is necessary when 
a project is not producing. 

One simple way to avoid negative evaluation of a good idea is to 
monitor the idea every step of the way. This monitoring Includes scoring 
the project just as carefully as one would keep score during a baseball 
game. When the product or project is not up to standards, something 
should be done immediately. 

There is nothing magic about a score card. Sometimes, Its biggest 
advantage is in stirring some managers into trying something that could 
have been done all along. Without the Impetus of the score card, this 
common sense and easy to Implement solution might have been overlooked. 
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LEARNING TECHNOLOGY 



As a result of the resource question, the cooperative educator Is 
able to benefit from a wide range of learning technology. These Instruments 
are given to students. These Instruments range from a high level of 
abstraction to a very concrete presentation. 

It Is customary to think of several levels of abstraction. The most 
abstract level refers to words, .The most concrete level refers to direct 
experience. 

The following two columns spell out this gradation: 
The first column refers to the level of abstraction. 
The second column gives an example of each level. 



LEVEL 



EXAMPLES 



Words 

Graphics 

Auditory Aid 

Still picture 

Video-tape 

and motion pictures 

Live TV 

Exhibit 

Field Trip 

Demonstrations 

Simulation 

Direct Experience 



Lectures, printed material 
Charts, diagram, graphs 
Recordings, tapes 
Photos 9 slides, slide films 
Documentation, animations 

Immediate, visual shows 
Display of objects 
Familiarization ' 
Lab Instruction 
Devices, role plays 
Guided 9 actual work 
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The difference between the degree of abstraction present in related 
instruction and in cooperative education is not always found in the level 
of abstraction. In general, related instruction is intended to give breadth 
to the limited specialization found on the cooperative job. In other words, 
even the related instruction connected with cooperative education should 
try to approach the concrete level. 



It is not unusual to hear students exclaim, "You need a CPA, an 
attorney, a semanticist, and a metaphysician to understand some of the 
explanations given by that professor. The highest grade in our course 
was achieved by a student who studied hieroglyphics," 

This exaggrated comment points out the fact that a high level of 
abstraction is not always the most desirable in related instruction. 

On-the-job training certainly gives a highly specialized and narrow 
taste of an experience. The job of related instruction is to give more 
breadth and overview. Attempts to avoid highly abstract terminology and 
concepts result in a bet'ter blending of on the job experience and related 
instruction. In this way, each is a natural extension of the other. 
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THREE EXAMPLES OF THE ALVIR APPROACH TO EVALUAHON 

In sunimary form, it is very easy to sum up this concrete approach 
to evaluation. Here are three typical rules that provide a good overview: 

1. Enunciate or write goals 

2. Verify a variety of yardsticks 

3. Recognize all learning opportunities 

RULE 1 : ENUNCIATE OR WRITE GOALS 

- Goals enable learners and teachers to compare the results of the 

learner before and after instruction. Goals furnish a constant reminder 

as to what is expected. When the temperature gets warm or the outdoors is 

more attractive than indoors, goals keep the learner on the track. Try 

to turn a general goal into a specific objective. This is one way to 

place* the goal out in the open for all to know and appreciate. 

Goals come in many different sizes and types. Here are three examples: 

KO or Knowledge Objectives refer 
to data. 

. PO or Performance Objectives refer 
to things. 

AO or Attitude Objectives refer 
to people. 

All three of these dimensions are necessary. The first time a 
cooperative educator writes out a goal, it is likely that only one of the 
trio'of KO, PO, or AO will be put down on paper. After this occurs, it is 
the task of the educator to balance the coopeifative educational program with 
the misfiing ingredients. 
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RULE 2: VERIFY A VARIETY OF YARDSTICKS 

It. is a commonplace to state that cooperative education evaluation 
must apply on-the-job standards. From the point of view of humanistic 
education, it is just as Important to insist upon the use of gains score 
when and where appropriate. This refers to the delicate balancing of job 
standards with growth standards. 

As far as timing is concerned, evaluation should be more than a pass 
or fail judgment. A certain amount of progress evaluation should be 
dncluded in every program. 

Evaluations and evaluation yardsticks come in many different sizes and 

types. Here are three examples: * 

KE or KNOWLEDGE EVALUATIONS refer to the 
barometer ability to predict impending 
changes and trends. 

PE or PERFORMANCE EVALUAnONS refer to the 
weather vano ability to spot how the wind 
is currently blowing on the cooperative 
education scene. 

AE or ATTITUDE EVALUATIONS refer to the 
thermometer ability to keep track' of 
human temperatures and emotions. 

The new generation of cooperative education student is more demanding 

and more knowledgable. As a result, this challenge Is even more interesting 

to the .educator who uses a wide variety of evaluation Instruments. 
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RULE 3: RECOGNIZE ALL LEARNING OPPORTUNITIES 

Cooperative education students must be able to learn from experience. 
Experience includes both success and failure « 

In addition to this ability to self -evaluate and self -motivate, the 
cooperative education student xmist be habituated to .incorporate the 
unanticipated. Many of the valuable learning resources for cooperative 
education cannot be neatly prescheduleo and anticipated. This activates 
the ability of the learner to be adaptive and creative in response to a 
constantly changing learning atmosphere. 

Resources come in many different sizes and types. Here. are three 
examples: \ , 

KR or KNOWLEDGE RESOURCES refer to hundreds 
of inputs of learning that can be conceived 
as mechanized or computerized, either in 
reality or in organizational format. 

PR or PERFORMANCE RESOURCES refer to all 
kinds of human thermostats that can react 
to changing conditions by stressing 
appropriate learner responses. 

AR or ATTITUDE RESOURCES refer to ESP 
(extra-sensory perception) that enables 
learners to feel or anticipate what is 
coming next even before it happens. 

All three types of the above learning resources are necessary. 

Different types of success have different causes just as different learning 

problems require different remedies. Not every teacher can anticipate 

every possible situation, but the resources provided should be plausible 

to the reasonably enthusiastic learner. 
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THREE IMPLEMENTATION CONCLUSIONS OF THE 
ALVIR APPROACH TO EVALUATION 

In summary form, it is easy to sum up the conclusions of this approach. 
Here are three typical conclusions that provide a good overview: 

CONCLUSION 1 

Content should determine course length, not vice versa, 
CONCLUSION 2 

Trade task analysis objectives should determine the content, not 
vice versa* 

CONCLUSION 3 

Even the' best content is seldom -sufficient and adequate for an 
indeterminate future • 

CONCLUSION 1 ; CONTENT SHOULD DETERMINE COURSE LENGTH,, NOT VICE VERSA. 
Because content will vary from one area of specialization to another, it 
is essential that course length also vary. Any cooperative educator who 
falls back upon such a magic formula as 144 hours or a three year six 
semester program is doomed to failure by an excess of rigidity. 

In any institution, when all the programs last the same length of 
time, one can reasonably suspect that something is wrong somewhere. In 
other words, every educational institution must have a place for a few 
short and quick programs. Similarily, in some technical programs, it 
would be a good idea to introduce the learner early in life to the concept 
of the 40 hour week. If students remain with the idea that 15 hours is 
the normal work week, such learners are going to be in for an unpleasant 
surprise. 



CONCLUSION 2: TRADE- TASK ANALYSIS OBJECTIVES SHOULD DETERMINE THE CONTENT, 
NOT VICE VERSA. It would be a misunderstanding of conclusion 1 to presume 
that past content is the guideline to course length. When conclusion 1 
asserts that content should determine course length, it is meant that the 
content of the technical task is more important than the content of previou 
lesson plans bearing the name of the trade specialty in question. 

The one reason why people repeat over and over the same program is 
simply to avoid red tape. Such conformists apparently have heard little 
about regionalization as an expression of cooperative education. Sometimes 
this adherence to formality is based upon the experience of educators who 
have found out that it often takes two yfears for minor modification. 

From a more practical point of view, many work study programs started 
ouL.as a money maker to allow the learner to stay in school. After a while 
these programs ended up as hybrids in order to justify existence in an 
academic setting. This is still another reason to go back to on-the-job 
standards to determine course objectives based upon actual task analysis. 

CONCLUSION 3 ; EVEN THE BEST CONTENT IS SELDOM SUFFICIENT AND ADEQUATE FOR 
AN INDETERMINATE FUTURE, Thus, even if there were once only one way to 
do it, any program that avoids change is forgetting about the future. 

On occasion, educators look to laws and statute!; as magic formulae 
or unchangeable alternatives. 

. The learner, as well as the educator, must make many assumptions, 
about the future. Is the past to be repeated? Is it probable that what 
worked today will work tomorrow? How far into the future can our long- 
range projections go without serious error? 

The answers to these questions require evaluation. In this context, 

evaluation is seen as a very specific way to improve learning success. 
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PART 2 . 
IMPLICATIONS DEVELOPED BY PARTICIPANTS 
AN OVERVIEW 

The following two pages are divided In two columns. 
Column 1 Is entitled PROGRAM EVALUATION EXPECTATION. 
Column 2 is entitled EVALUATION SUGGESTION- 

These two pages provide a resume of what the participants expected; 
this is provided in the column entitled PROGRAM EVALUAnON EXPECTATION. 

The second column entitled EVALUATION SUGGESTION summarizes what the 
participants brought to the conference in the way of suggestions. 

As an exercise to the reader, it is suggested that the reader go 
through both columns with a pencil in hand. Whenever the reader discovers 
an expectation or suggestion that is clear, each clear item is marked with 
a C. Whenever the reader comes across an expectation or suggestion that 
is unclear, this item is marked with UC. 

This rapid survey of the following two pages will help the reader 
clarify the initial understanding of part 2. Similarly, this initial 
analysis will give the reader a better idea of what participants at the 
evaluation conference did with the presentation made by the present author. 
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APPLICATIONS DEVELOPED BY PARTICIPANTS 



PROGRAM EVALUATION EXPECTAHON 



1. Document learner success due to 
staff development (where are 
students helped?) 

2. Pinpoint convincing and 
uncomplicated evidence that 
documents program success beyond 
opinion and hearsay 

3. Cost-Justify evaluation 
expenses 



4. Specify "progress" advantages of 
coop over conventional 

5. Measure progress levels in an 
office environment as opposed 
to simulation in a classroom 



6. Decide where to evaluate? Whole 
program, a ispecific part or area? 

?• Decide: (a) Who would do the 
evaluation? 
(b) What kind of followup 
will be done? 

8. Specify criteria for evaluation 



9. Idcatify available ready-made 
tools or instruments and the 
range of costs connected with 
each alternative 

10. Refer to results and scientific 
surveys that demonstrate the 
usefulness to students 

11. Factor out the natural selection 
effect to isolate how much succes 
is due to the program 



EVALUATION SUGGESTION 



1. 

a. Write it down (Plan or Goal) 

b. Stick to it (Program) 

2. Useja consistent form, format, 
or booklet 



3. Administer "progress checks" in 
each unit-function before it's 
too late, prior to one final 
evalxiation 

4. Utilize authentic work even in 
simulation (e.g., type wills in 
a law office, not invoices) 

5. React to student evaluation 
criteria: What value is this 
course or experience? Should 
a change be made? 

6. Gather consumers data (I.e., 
from the student) 

7. Do something different as k 
result of the evaluation: Act 
on the recommendations verbally, 
operationally, and structurally 

8. Elaborate clearly what is being 
researched instead of hinting 
at it 

9. Funnel the evaluation report 
to the right person 



10. Offer experiences With style 
in place of courses. Style is 
how one does things 

11. Realize the employer will 
provide academic evaluation of 
the learner 
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PROGRAM EVALUATION EXPECTATION 



12 • Give employ.ers more than just the 
leftover time in schedule 

13. Define program success in 
realistic terms 



14. Start up an evaluation program 



15. Rank the evaluative criteria in 
terms of importance (from most 
important downward....) 

16. Determine the difference between 
the plan (What I wanted to do?) 
and the product (What 1 did?) 

17. Gain an overview of what, is 
happening 



18. Catch up on the current "in" 
words 



19. Design for diversity 



EVALUATION SUGGESTION 



12. Get a sense of direction via 
shared plans and discoveries 

13. Grow on the job as well as 
function under constant 
supervision 

14. Study feasibility of: 

(a) Calendar and curriculum 

(b) Student participation 

(c) Employer participation 

(d) Coordination staff 

^ (e) Rules and Regulations 
(f) Program funding 

15. Be fair, admit biases and gather 
the facts 



15, Intermingle the evaluator and 
person evaluated in f common 
participating effort 

17. Pick the best questions and 
inquiry tools to get the most 
accurate results 

18. Admit the problems, without 
panic, but with a step forward 

19. Define operationally: function 
for function, dollar for dollar, 
goal for goal 
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After examining the above two columns, the reader is now ready for a 
detailed analysis. The reason for this detailed analysis is to make sure 
that the program evaluation expectations and suggestions are used to 
document learner progress. 
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ANALYZING EXPECTANCIES AND SUGGESTIONS ON 
FOUR DIFFERENT LEVELS 



Throughout this document, evaluation is considered an effective means 
of improving program success. This success is always measured in terms of 
learner accomplishment. 

In the area of inservice education, several distinctions are in order. 

1. Very often, classroom teachers are tha learners for the 
inservice education. 

2. This would reasonably lead the instructor of inservice education 
to conclude that the success of the teachers attending these 
sessions is the best way to measure the success of each session. 

3. This over-simplification forgets that the real recipients of 
inservice education are the learners under the tutelage of 
the teachers instructed during a staff development inservice 
session. 

All of these elements are important. There are many ways to 
systematize the levels of expectancy of the teachers in attendance. , 

The message chosen here is four level analysis: Level I refers to 
the fact that the teacher acquires a new competency at an inservice staff 
development training session. This competency could be the ability to 
evaluate more effectively. 

Level II refers to the fact that the teacher tries to exercise this 
newly acquired competency back home in a school or classroom situation. 
Success on Level II means that the teacher has made an honest effort to 
improve • 

Level III is the situation wherein a teacher makes an honest effort 
to exercise the competency and to adapt the competency to local conditions. 
This sometimes includes overcoming barriers and obstacles that vary from 
one setting to another. This could even mean that a teacher who applies 
a new competency in one wcy in a specific school will be obliged to change 
the application in another school in response to the local environment. 
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- Level IV refers to the fact that a teacher with a new competency is 
able to document lear.ner success due to this competency. Success at this 
level is the desired end product of all inservice staff development. When 
the educator is able to produce evidence of benefits to learners, the 
benefit analysis of education has gone the complete circle. In such a 
situation, educational dollars are well spent. 

The reader is invited to look at the following page entitled, LEVELS 
OF EXPECTANCY • These four levels of expectancy are illustrated from the 
preceding section on participant expectations. 

The following page entitled, LEVELS OF SUGGESTIONS FROM PROGRAM 
EVALUATORS is a similar analysis performed on the suggestions submitted 
by participants, . 

Looking at both of these four level analyses will help the cooperativ 
educator evaluator recognize the importance of all four levels. 

While examining the following two pages and the four level analysis 
contained therein, the reader is urged to recall the fact that this 
material comes from a random sample of cooperative educators. There x 
are many other equally good possibilities. The point here is to avoid 
getting bogged down in such a large number' of low level possibilities 
that the evaluator loses sight of documenting impact on the learner. 
As a result of evaluation, learners are expected to improve if the only 
result of evaluation is grading and sorting of learner?, the evaluation 
that results from this is certainly missing something. This missing 
component is the impact on learners. 
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LEVELS OF EXPECTANCY 



LEVEL I t Acquire Competency 

1. Specify progress advantages of this competency 

2. Identify available ready-made tools and instruments 

3. Refer to results and scientific surveys that demonstrate 
usefulness to students 

4. Define program success in realistic expectations and language 

5. Rank evaluation criteria in terms of importance (from urgent 
to routine) 

6. Gain an overview of what is happening 
?• Catch up on the "in" words 



LEVEL li t Exercise Competency 

1. Decide where and what to evaluate 

2. Specify criteria for evaluation 

3« Factor out the natural selection effect to isolate how much 
success is due to the program 

4. Start up evaluation program ' 

5. Design for diversity 



LEVEL Ill t Adapt Competency 

1. Cost-Justify local evaluation expenses 

2. Decide who will conduct evaluation 

3. Decide what kind of followup will be done 

4. Give employers more than just the leftover time in the schedule 

5. Involve non-project staff in most operations 



LEVEL IV t Document Learner Success Due to Competency 

1. Document instances of where learners are helped 

2. Pinpoint convincing and uncomplicated evidence of learner success 

3. Measure progress levels in a free enterprise work environment 

4. Determine the difference between the plan and the learner. product 

5. Correlate evaluation by employer supervisor and school supervisor 

6. Respond to learner-operated program evaluations 



This four level analysis of expectancies held by cooperative 
educators is paralleled by the next page which make a four level analysis 
of suggestions originating from these same educators* 

Er|c 31 



20 



T.B\nr.T.S OF S itnGESTIONS 
from Program Evaluators 



LEVEL I : Acauire ComEitenci 

1 use a consistent approach (form, format procedures) 

I rab:rate ana discoveries 

3,. Get a f calendar and curriculum 

4. Study feasibility of (a) c^^^^^^ participation 

(c) employer participation 

(d) coordination staff 

(e) rules and' regulations 

(f) program funding 

LEVEL II: Exercise Competency 



1. 
2, 
3, 



Exercxse '■"'"h'^'- — 

. 1 <!Hck to the plan till completion 

write the plan down simpl^^^ 'eJ^n in simulation 

Utilize authentic work tasKs ^ve 

most accurate results 

l^VELIII: AdaEt ComEetencir , 

.uorks long before the final evaluation 
1. Administer progress checks long be o 

• 0 React to learner evaluation criteria evaluation 

\: To southing ^'"tC^L^tTnf « l^n/atlons to th. right P«son 

7 Define operationally, tunctio 

8. rift«°toTndlvldualsa«.g the participant.- 

IV: DocsESt SHCca^ SSe H C«^^ 

~" ^1,^ iob as well as functioning under 

1 Pinpoint growth on the job as wei.i 
constant supervision 
>3- r^tfr^Il^trtLnrat^tr aSTa .parso. evaluate. In a oo-on 
r."°-"Sy4Su for the benefit of the Individual 
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REACTING TO ALL THIS MATERIAL 

The basic idea of part 2 is that participants at a cooperative education 
evaluation conference reacted to part 1 of this document by producing 
individualized plans of attack. In a similar manner, readers of part 1 
and part 2 are invited to adapt this material for local implementation. 

In order to facilitate the development of local solutions by local 
staff for local problems, the following page entitled FOUR LEVELS OF 
COMPETENCY (ADVISOR CENTERED) is presented. 

This four level analysis was developed by an educator who had finished 
reading part 1 and the preceding pages of part 2. 

Each reader of this document is invited to carefully study these four 
levels of competency with the intent of' doing the same thing for the local 
program. Any educator who is able to apply this material to the local 
situation is thereby qualified to turn evaluation into a tool for improving 
effectiveness. 

Of course, no claim is made that the first effort of four level analysis 
will succeed. However, any educator, who notices that one or two of the 
levels is empty, is thereby able to balance out the task. 

Obviously, level IV,in which learner success is documented, is the 
most important. However, the other levels must not be neglected since 
each of these levels contributes to the eventual success of the learner. 
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I FOUR LEVELS OF COMPETENCY 
I Advisor-Centered | 



I - Possess Competency (Competency here refers to teacher-administrator 

ability in cooperative education)* 

A. Administrative Interest 

B. Coordinator Qualification 

C. Parent Orientation and Permission 
Active and MeaningfulAdvlsory Committees 

E. Academic Teacher Support 



II - Implement Competency 

A. Administrative Support 

B. Coordinator Performance 

C. Course Content 

D. Student Selection 

E. Use of Training Outlines 



III - Adapt Competency Locally 

A. Coordinator Time Available for Supervision 

B. School Physical Teaching Facilities 

C. Survey of Community Facilities 

D. School Guidance Cooperation in Scheduling 

E. Job Stations 



IV - Document Learner Success Due to Competency 

A. Job Relevance to Student Needs 

B. Student Performance on Job 

1. Evaluation by School Supervisor 

2. Evaluation by Employer Supervisor 

C. Fulfillment of Learner-Centered Program Objectives 

D. Learner Followup after Completion of Program 



Each reader is asked to reflect upon local situations in order to 
come up with a similar analysis based upon the four levels of competency. 

This local adaptation will make the approach to evaluation herein 
stressed more practical and meaningful* 
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USING MATRIX ANALYSIS TO SWITCH OVER FROM 
TEACHER-CENTERED TO LEARNER-CENTIRED EVALUAHON 

The diagram on the following page contains 9 components. Each of 

these components Is Identified by a two letter code. 

K Is the Indicator of the knowledge or data domain 

P Is the Indicator of the performance or things domain 

A Is the Indicator of the attltutde or people domain 

0 Is the Indicator of the objectives, goals, or targets 
dimensions 

E Is the Indicator of the evaluations, tests, or measurement 
dimensions " . 

R Is the indicator of the resources, technology, or 
methods dimension 

Thus, KO stands for knowledge objectives, PE stands f^ performance 
evaluations, AR stands for attitude resources, and so forth. The preceding, 
page entitled FOUR LEVELS OF COMPETENCY: ADVISOR CENTERED was based upon the 
functions of the teacher. The following page entitled MATRIX ANALYSIS: 
LEARNER CENTERED Is based upon the requirements of the students. In many^ 
ways, the verbs on both pages are similar. Yet, In the last alTalysis, the 
verbs found on the matrix analysis tell what the learner will do. Thus, 
matrix analysis Is on level IV which tries to document in one way or another 
all types of. learner success. 

The reader who has gone through the MATRIX ANALYSIS Is In a good 
position to try to develop a matrix for local problemsand priorities. 
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Coop Education 
MATRIX ANALYSIS 
(Learner-Centered) 



K • Knowledge (Data) 



P » Performance (Things) 



A = Attitudes (People) 



KO 



PO 



AO 



Learn through work 



Pinpoint up-to-date data 



Develop realistic 
expectations 



Experienc e the actual 
operation of the world of 
work 

Achieve measurable 
productivity 

Put principles into practice 



Develop proper work 
attitudes 



Mature and 

Function as an adult 

Appreciate values of free 
enterprise system and 
economic responsibility 






Succeed on related 
classroom instruction exams 

Talk about enjoyable and 
profitable work and work- 
related experiences as well 
as problems 

Specify a rational and 
meaningful personally 
chosen plan 



Hold a job by meeting 
expected performance 
standards with a satisfactory 

Progress on the job 

Fulfill learning agreement 
ans work agreement 

Gain seniority on the job 
and productivity-based 
salary increments 



Satis fy employer 

Follow instructions 
willingly 

Describe convincingly 
individual commitment as a 
result of experience. 

Self-evaluate personal and 
occupational goals in ligh 
of first hand experience 






Discuss off-campus require- 
ments and details 
Choose voluntarily related 
reading material in news- 
papers, magazines, and 
media 

Confront new ideas , 
experiences, and 
responsibilities 
Participate actively in 
workshop type sessions v^ith 
a variety of peers in a 
range of work occupations 
Visit nearby businesses 
and industries 



Work for participating 
employers 

Keep a work record 

List accomplishments 

Write out the learning 
agreement and the work 
agreeijient as a "training" 
tool and contract 



Get along with peer group 
co-workers 

Adapt to adults other than 
teachers and direct 
supervisors . 

Keep a diary of incidents 
React imaginatively and 
personally to work 
experience and attitudes 
encountered 



LOOKING BACK AT FOUR LEVEL ANALYSIS 
AND MATRIX ANALYSIS 

Four level competency analysis allows the advisor -centered approach 
to gradually move up to level IV where the learner is the primary 
interest. When level IV Is reached, the matrix comes into play. 

The matrix allows the educational community to pinpoint learner- 
centered objectives , evaluations , and resources in the knowledge , 
performance , and attitude domain. ^ 

This is why the matrix has been, called a one page planning tool. 
SiUiilarly, the matrix is a collection and classification of things the 
learner can do. In other words, the matrix helps educators define 
operationally the expectations made on learners. 

CONCLUSION 

When the evaluator has begun to think in terms of the learner, the 
evaluator is ready to consider evaluation as an effective tool to improve 
learner success. Evaluation improves learner success by documenting the 
impact of teacher and program competency on the learner. 

The approach herein advocated is simply an analytical, organized, 
and balanced approach to this evaluation process. 

Evaluator s who employ this approach can take the best of what they 
are currently doing- and combine these best elements with the best elements 
of other evaluators of similar programs. In this way, it is a spur to 
greater progress. 
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LEARNING ENVIRON^ENTS 
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LEARNING ENVIRONMENT 1 ; ACTIVITY-BY-ACTIVITY EVALUATION (OBJ-1) 

In order to provide an activity-by-activity evaluation that identifies - 
objectives,' time period, resources, personnel, and success of outcome, the 
follox/ing procedures may be employed* 

1. Identify the major activities of the inservice education program. 

2* Attach to each major activity in the program a short list of two or • 

three objectives. Each of these objectives should begin with a verb. 

The subject of each of these verbs should be the participant attending 

the inservice education program. 
.3. Provide an evaluation form that lists, among other items, the objective 

number and a chance to rate it from very good, good, average, poor, or 



4. Make sure that the above procedures are carried out with adequate 
explanation provided to participants as to what is desired in this 
type of evaluation. 

This type of evaluation is done activity-by-activity on a very detailed 
level. This type of activity will provide a number of counts that can 
provide a session«*by**session evaluation. 

LEARNING ENVIROI^IENT 2 : IDENTIFY POSITIVE AND NEGATIVE FACTORS (OBJ-2) 

After the activf.ty-by-activity evaluation stressed in Learning Environment 1, 
the program director is able to form an overviev/. 

This overviev? requires 'some simple way of keeping score whether by 
percentage or by total points. 

For example, a typical overview could look like the following two columns. 
Column 1 is an identification of the session. Column 2 is an identification 
of the percentage of successful achievement of objectives. 



It is the task of the project director to. identify activities Ott organizational 
mechanics which contributed or detracted from the overall objectives. Looking 
at the two columns above provides a very simple format to identify conimon 
factors that were either negative or positive. 



Inferior. 



Session Identification 



Percent of Success 



Session 1 
Sessions 2 
Session 3 
Session 4 
Session 5 



95% 
60% 
40% 
90% 
30% 
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After careful analysis of. the reasons for the outstanding success of 
session 1 and 4, it is possible for the program director to conclude that 
success was due to: 

Teacher personality 
Media adaptation 
Audience participation- 
Question and answer period 
Open discussion 
Group interest in the topic 

Appropriateness of topics to the needs of the audience 

These positive common factors should be stressed in future workshops 
as elen'.ents that contribute to success. These factors of success should be 
pointed out to the parti 'iipants and instructors in order to reinforce the 
positive influence of these activities or organizational mechanics. 

In addition to finding out common factors of success, the program 
director should look for weak links. This might mean that certain sessions 
such as 2, 3> and 3 were unsuccessful because of: 

Too much reliance on the lecture method 

A very dull presentation 

Nothing new was said 

The audience remained silent 

Audience needis were not addressed 

The topics covered were poorly presented 

After looking at these specific negative factors on an activity-by- 
activity evaluation, the program director is able to pinpoint specific 
things that should he eliminated the next tiT^e around. Sometimes, an^ 
activity-by-activity analysis can identify activities or structures that 
can be changed immediately on the spot. This on line improvement will 
contribute to the overall success of the conference. 

Learning Environment 1 has stressed a MICRO approach to evaluation. 
Learning Environment 2 has stressed a MACRO approach to evaluation. The 
micro approach zeroes in on details. The macro approach tries to form an 
overview of the inservice education program. Both elements are necessary 
for a well balanct-d evaluation. * • 

LEARNING EMVIROMI'IENT 3 t >lA6Ek (OBJ-3) 

It would be appropriate for project directors to become aware of some of 
the current literature on management by objectives and leadership by 
objectives, a few names would come to mind: M^ger* Pophara, and others. 

These authors should be consulted either in textbooks or in journals in 
order to provide project directors with specific exnu^ples of how to 
evaluate a conference according to prespecified objectives. 

One simple way to start would be to go to the ERIC collection and the 
attached RxE (Research in Education) index. Looking up topics of interest 
in thiu way would provide the project director with up-to-date informntion 
on a wide variety of approaches. After having looked at a wide variety of 
alternative activities and orgnnis^ational mechanisms, the project director 
would be in a better position to individualize. . 
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LEARNING KNVIROMMEiNT 4 : IMPORTANT ATTITUDES (OBJ-3) 

It is Important for the inservice 'education program instructor to realize 
that a small percentage of teachers present among the participants are there 
principally for academic credit or the small stipend. The inservica program 
must be 'organized in such a way as to remotivate these teachers to participate 
to acquire a new competency. 

A new competency can be acquired on four levels within the framework of 
inservice education programs: 

LEVEL I: The. teacher acquires a new competency . 
LEVEL II: The teacher applies this new competency back 

in the home school 
LEVEL III: The teacher adapts this competency to local 

implementation circumstances and difficulties 
LEVEL IV: The teacher documents the effectiveness of 

this new competency with specific texamples of 

impact on student learning 

In this way, an inservice instructor can take teachers v/ho are present for 
a wide variety of motives and remotivate these teachers into a striving 
after new professional competency. This is a challenging task, but the 
reward is worth the effort. . 



LEARNING EN^7IR0IvI>1EyiT 5 ; COUNTABLE RESULTS (0BJ'-4) 

One of the secrets of evaluation is to know what to count. 

This means that the successful participant in inservice education programs 
must be able to go back to the home school with a definite idea of countable 
student successes that document the importance of the teacher competency 
acquired during a workshop. 

For example, a teacher just ended a workshop on employability profiles. As 
a result of this workshop, the teacher was able to divide a printing course 
into five employability areas: 

1. Operating the offset camera 

2. StrippJ.-^.g the negatives • . . 

3. Making the plates 

4. Operating the press 

5. Binding printed material 

With such an approach, the teacher was able to count partial as Veil as 
complete succei^ses. A partial success would be an in.stance vrherein a 
student was able to gain entry level employment in one of the five major areas 
A complete success will be an instance wherein a student was able to obtain 
entry level employment in two or three of.thn major offset printing jobs. 
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The ability to count visible and measurable success in an important point 
in the type of followup necessary to measure the impact of Jaservice 
education programs. 



LEARNii'iG ENVIRONMENT 6 1 LOCAL PLANS FOR CLASSROOM TEACHERS (OBJ-5) 

Some teachers go to a workshop with the anticipation of coming home with a 
pref abricr : ed plan that requires very little investment of local planning 
time. Such prefabricated plans do not always work out. 

The first step is for d teacher to acquire several examples of what 

-has been found to work in a variety of individual circiamstanccs. 

The second step is for the teacher at the ^workehop to preview some of the 
first implementation steps that can be taken in light of the workshop period. 

The third step at the workshop is for the group to preview local implementation 
difficulties. In some places, the schedule will be different, the students 
will be different, the barriers will be different, the problems will be 
different, and the reactions of the staff will be different. 

In anticipating the necessity for local adaptation, the educator at an 
inservice education program will be in a better position to overcome local 
implementation difficulties. 

After this has been done, the teacher should try to come up with countable 
examples that document learner success due to competency possessed by the 
teacher. 

LEART:xNG ENVIR0N>!KNT 7 ; FOLLOWUP EVALUATION INSTRUim^TS (OBJ-5) 

It is highly desirable to find out what teachers did as a result of 
participating in inservice education programs/ 

There are some advantages in using a single instrument to tabulate the 
followup results by comparing one teacher with another. However, room 
should be left to give each person enough freedom to use individual 
ingenuity. 

Sometimes, individual ingenuity is foimd expressed in a homemade follox\nip 
instrument. The important point here is to make sure that the local 
followup instrument provides data that can be used by other educators 
in different schools and in different circumstances. 

One simple followup technique is to use a telephone sutvcy. This enables 
the project clirectior to contact a selected sample of workshop participants 
in order to ask such questions as: 

1» In what specific ways have you been able to use the 
knowledge, skills, or attitudes picked up at the 
inservice education program you recently attended? 

2. How hr,ve you been able tu document learner success reldted to 

conipetencies acquired at the recent inservice education workshop? 

41 
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LEARNING ENVIRONMENT 8 : BASELINE DATA INSTRUMENT' (PRETEST-1-2-3) 

It is the responsibility of the project director to come up with some 
type of an instrument to pinpoint precisely where the participants 
are at the beginning of the workshop in relation to the objectives of 
the workshop • 

The gathering of such baseline data avoids the situation wherein most 
participants are exposed during the first few. days to things they 
already know. 

Possibly, such a baseline data* check could be a duplicated instrument 
sent along with the application for the inservice workshop. 

This baseline data instrument should include knowledge evaluation (KE), 
performance evaluation- (PE), and attituc^« evaluation (AE). 

KE refers to things the pa. Iclpant already 
knows or has previously reau about. 

PE refers to background experiences and 
present skill levels of the participants in 
the areas under study in the workshop. 

AE refers to sounding out the feelings 
and values of the participants with 
regard to the objectiv^r-s . yardsticks, 
and procedures of the workshop. 

This kind of a check on where the participants are should be done several 
weeks or months before- the workshop. This type of diagnostic preassessing 
enables the workshop director to plan a program that is individually suited 
to the participants. 

LEARNING ENVIRON^yENT 9: DIFFERENT EVALUATION PERSPECTIVES (PRETEST-6) 

Experience seems to indicate that even the best workshop will evoke » nuriier 
of positive and negative comments on the part of participants. Whenever 
the conimen, s are skewed too much in either the negative or positive direction, 
something is out of order. 

Without going to either extreme, the negative extreme of pressing participants 
to find something wrong with an excellent workshop or the positive extreme 
of forcing participants to find something good about an inferior workshop, 
the evaluation should include both negative and positive observations. 

One simple way to do this is to stress at least three different perspectives. 

Perspective 1 could be the perspective of the workshop director and his 
objectives. This simply means that the vrorkshop director keeps score and 
explain ^ how his ov-^irall evaluation was arrived at* 
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The second perspective is that of the, participants and their objectives. This 
simply means that each participant spells out the major anticipation for the 
workshop as well as the major results of this workshop when viewed from the 
Individual's point of view. 

The third perspective is that of outside evaluators. From a research and 
evaluation point of view, this would mean pinpointinj; things that can be 
duplicated elsewhere at a reasonable cost with good expectations of success. 
From a management point of view, this would mean deciding whether or not to 
conduct the institute the next year at the same site or at a different site, 
with the same workshop personnel or with different individuals. 

The workshop director who is aware of these 3 different perspectives as v;ell 
as a number of other possible differing expectations is in a good position to 
provide the type of evaluation that documents the overall impact of the workshop 
In contributing to educational progress. 

LEARNING ENVIRONMENT 10 ; REALISTIC EXPECTATIONS (PRETEST-1-2-3) 

The objectives proposed for a specific workshop should be realistic 
expectations. This means that the budget, the time available, the 
instructional personnel, and the participants are able to accomplish the 
prespecifisd objectives in the plan put forward. 

As far as knowledge is concerned, two or three days should be more than 
adequate to convey the basic information and conceptual framework. 

When a workshop has objectives that go Into the performance or attitude 
domain, two or three consecutive days are normally adequate to the task. 
Sometimes, this type of a pro£ram can be arranged to have two or three one« 
day sessions several weeks apart. The time between sessions allov?s both 
staff and participants to readjust individual activities in order to 
•achieve prespecificd objectives with greater success. 

The difference between a realistic expectation and an impossible dream depends 
upon a number of variables. A thorough av:arenass of the competencies of 
participants can have much impact here. Given the right project director, 
staff, and participants, almost any objective can be achieved if the. budget, 
timetable, and resources are adequate. 

However, the typical situation is that only so much can be achieved In a 
certain period of time given the constraints of the participants and staff. 
A- realistic expectation requires an accurate and objective assessment of what 
can be done in a "given period of time. 
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LEARNING KNVIRONMENT 11 SPECIFYING WIIAT IS WAITED (OSJ-1) 



It is sometimes interesting to take the view of an outside observer 
watching t^o different mentalities plan for an inservice workshop. 
One type of mentality is constantly asking and answering the question, 
*\that is wantedj" This type of person zeroes in on objectives, purposes, 
and benefits© The results of such an inquiry are usually specific and 
measurable* 

A second type of mentality is constantly asking and answering in a 
dozen different ways the question, "Who is going to teach this workshop?" 
This type of person possibly has a pal or crony in mind© The difficulty 
arises from the fact that the well qualified associate may be mor^ on 
the mind of the planner than the needs of the typical participant for 
vhom the workshop is being designed© 

There is nothing the matter with either of the above questions o On the other 
hand* the tx;o questions are not -equivalents The t;i7o questions are not of equal 
value# The first question to be asked in planning a workshop is, "What 
Is V7anted?" The nest auestion, which must come after the first question 
has been answered, is/'t^ho will teach in the workshop?" 

The practical advantage of asking the first question first is obviouly found 
In the ability to come up with a large number of ansv/ers for the second 
question if the objectives of the workshop are precisely spelled outo 
When a specific desired product of a v/orkshop is clearly identified, 
the personnel invo2.vcd are able to produce this result with a number of 
different people, at a number of different price tags, in a number of 
different ways, and in a variety of different circumstances. Anyone 
who asks the second question first is like somebody x-yho wants to go from 
city X to city Y but only by a pecif ic road. If the road is chosen 
before the destination, it is quite likely that a new express highway 
\9ill never be utilized by a planner who chooses the path before the 
destination. 



LEARNING KmaRONMRNT 12 DISTINGUISHING BETI^TIEN A CATALOGUE AND A BLUEPRINT 

(OBJ-1, OBJ-2) . 

The term LEARNING ENimiONMENT has been used in place of guidelines, 
criteria, or directives for the evaluation techniques herein suggestedu 
A catalogue lists a large ninril)er of items from which each individual 
%^ill choose one or two appropriate tools© A blueprint lists everything 
that must be included in order to make the construction complete. With 
these definitions in mind, this collection of learning environments 
Is more likjS a catalogue than a blueprint. 

As In pro football, this collection is like a play book. It is a good 
list that gives a wide variety of alternatives. Each project director 
is to consider himself an evaluation couch who will choose one play 
or strategy at a time, Tlie exact choice will be made to match the 
local team available and other individual constraints. 
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The inexperienced project director will feel it obligatory to cover 
every learning environment in this collection^ The hesitant project 
director might even feel it necessary to regtirgitatc this catalogue 
of evaltiation examples with only a few words changed to avoid total 
plagarism. Both of these errors boil down to confusing this catalogue ^ 
with a detailed blueprint « 

This collection encourages project directors to think seriously 
about evaluation,) The proof and result of this serious thinking is 
to be a documented plan. v7oirked out by each individual workshop director o . 
This workshop director will spell out individual objectives and ta^^ets* 
In order to make evaluation more objective, each workshop director will 
spell out a number of acceptable yardsticks with which to me^lsure progress o 
Some yardsticks will be original, others will be borrowed, and others 
will be adaptations or combinations of successful strategies fotmd 
elsev7here. 

With this perspective, txiy hypothetical case presented in any learning 
environment is to be interpreted as an example rather than ^3 a 
specif icat ion 0 . This example is intended to show v;hat CTOuld be done 
In order to give a concrete example in place of abstractions* Obviously, 
the workshop director who would copy this example detail for detail 
would be manifesting a certain amount of incompetency since it is 
difficult to assui\ie that local situations would exactly parallel the 
circumstances under x^hich this example was developed^ 



LEARNING EWIRONMEOT 13 

Here is where each reader wites in an idea, strategy, or value 
not cited above » , 
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